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(^BEGIN^ 

READ IN OR BUILD 
A SYNTHETIC SNAPSHOT 



USING THE SYNTHETIC SNAPSHOT, 
BUILD A DEFAULT IMAGE 
AND STRUCTURAL MODE 



USING THE GEOMETRIC DATA 

IN THE PACKAGE LIBRARY 
TO BUILD A GEOMETRIC MODEL 



INPUT A REGION OF INTEREST ON 
THE BOARD THAT SHOULD 
CONTAIN THE COMPONENT 



MATCH THE IMAGE AND STRUCTURAL 
MODELS TO THE REGION OF 
INTEREST TO DECIDE 
IF THE COMPONENT IS PRESENT 



YES 




USE THE ROUGH LOCALIZATION IN 
THE LAST STEP AS A SEED FOR THE 
GEOMETRY MODEL GEOMETRY 
MODEL WILL FIND THE EXACT 
BOUNDARIES OF THE COMPONENT 






* 




ROTATE THE IMAGE DELINEATED 
BY THE BOUNDARIES TO OBTAIN 
AN IMAGE OF THE COMPONENT IN 
THE DEFAULT ANGLE 








OBTAIN 'CROPPED" IMAGE OF 
THE PART (i.e. NO BACKGROUND) ^ 



FIG. 3A 

FIG. 3 
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STRUCTURAL AND IMAGE MODELS 
ARE IMPORTED AND APPLIED TO CROPPED, 
PASTE, PLACED AND BARE EXAMPLE ROIS 



REPLACE THE DEFAULT QUALITATIVE AND 
QUANTITATIVE PROPERTIES OF THE STRUCTURAL 
MODEL WITH VALUES LEARNED BY APPLYING THE 
STRUCTURAL MODEL TO THE FOUR IMAGES 
(CROPPED, PASTE, PLACED AND BARE) 



REPLACE THE DEFAULT ATTRIBUTES OF THE 

IMAGE MODEL WITH VALUES LEARNED 
BY APPLYING THE IMAGE MODEL TO THE FOUR 
IMAGES (CROPPED, PASTE, PLACED AND BARE) 



REPLACE THE DEFAULT GEOMETRIC PROPERTIES OF 

THE GEOMETRY MODEL WITH VALUES LEARNED 
BY APPLYING THE GEOMETRY MODEL TO THE FOUR 
IMAGES (CROPPED, PASTE, PLACED AND BARE) 
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IMPORT BOARD TYPES (BARE, PASTE, 
MULTIPLE PASTE) PART PLAN 



BUILD DEFAULT MODELS FOR EACH PART TYPE 98 



FOR A PARTICULAR TYPE OF PART, 
FOR EACH REFERENCE DESIGNATOR 
ASSOCIATED WITH THAT PART 
TYPE PERFORM THE FOLLOWING STEPS: 



USE THE DEFAULT IMAGE AND STRUCTURAL 
MODELS TO DETERMINE WHETHER 
THE PART IS ON THE BOARD 



FOR ONE REFERENCE DESIGNATOR TRAIN 
THE IMAGE AND STRUCTURAL MODELS ON THE 
IMAGES ON THE BARE, PASTE AND PLACE BOARDS 
IMAGES ASSOCIATED WITH THAT DESIGNATOR 



USING THE TRAINED MODELS, INSPECT OTHER 
REFERENCE DESIGNATORS ASSOCIATED WITH THE 
PART ON THE BARE, PASTE AND PLACE BOARDS 



TO 108 

r-\r* a a 

r i v_j . tn 

Fib 4 
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FROM 106 _ ^ ■.. 
Fi e . 4 A T 



DETERMINE HOW WELL THE TRAINED MODELS CAN 
DISCRIMINATE BETWEEN TRUE POSITIVES (PART IS 
PRESENT) AND TRUE NEGATIVES (PART IS ABSENT) 



OVER ALL REFERENCE DESIGNATORS, 
CHOOSE THE ONE SET THAT GIVES THE 
MODELS THE BEST DISCRIMINATION ABILITY 



IF NO ONE SET OF IMAGES/TRAINED MODELS 
GIVES GOOD DISCRIMINATION ABILITY USE 
A CLUSTERING ALGORITHM ON THE MODEL 
OUTPUT SCORES OVER TRAINED MODELS ON ALL 
REFERENCE DESIGNATORS TO DETERMINE 
HOW MANY EXAMPLES ARE NEEDED 



CHOOSE THE "N" SETS OF REFERENCE 
DESIGNATOR EXAMPLES WHERE 
N IS THE NUMBER CALCULATED 
BY THE CLUSTERING ALGORITHM 



CREATE, TRAIN AND SAVE IMAGE AND 
STRUCTURAL MODELS FOR EACH 
"CANONICAL" REFERENCE DESIGNATOR SET 
EXAMPLE FOUND IN THE STEP OF CHOOSING THE BEST 
N REFERENCE DESIGNATOR EXAMPLES 




Fi e. 4B 
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FOR A PART TYPE, LOAD IN THE MODELS 
. STORED IN THE PART PLAN 



FOR A REFERENCE DESIGNATOR THAT IS 
ASSOCIATED WITH THAT PART TYPE, GET A 
CROPPED IMAGE (ROI) OF THE PART 
AND ITS SURROUND AT THAT 
LOCATION ON THE BOARD 



CHECK TO SEE IF THE PART IS ABSENT BY LOOKING 
FOR FEATURES THAT PRESENT AT THAT 
REFERENCE DESIGNATOR ON THE 
BARE AND PASTE BOARD EXAMPLE BOARDS 
AND ABSENT ON THE PLACED BOARDS 




COMPUTE THE ANGULAR FEATURES 
IN THE ROI. DETERMINE THE MOST 
COMMON NON-ZERO ANGLE (THETA) 



v YES 


STOP AND DECLARE 




PART ABSENT 



APPLY THE IMAGE MODEL AT THE PARTS 
EXPECTED ANGLE AND THETA. 
COMPUTE A PROBABILITY THAT THE 
PART IS PRESENT AND RETURN 
THE MOST LIKELY LOCATION 
<x1,y1> AND ANGLE THETA2 



TO 146 
ROT-6B ^ 

no. GA 
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FROM 144 




USE THE STRUCTURAL MODEL 

TO VERIFY THAT THE PART 
IS PRESENT BY SEARCHING IN A 
SMALL AREA AROUND <x1 ,y1 > 
ASSUMING THE PART IS ORIENTED 
AT THETA2. 



b 




v NO 






f-150 


APPLY THE STRUCTURAL 
MODEL TO THE WHOLE ROI 
AT THE EXPECTED ANGLE 
ANDALSOATTHETA 





YES s 


156-v 




THE GEOMETRY MODEL 
USES THE POSITION 
AND ANGLE ESTIMATE <x2,y2> 
AND THETA3 TO EXACTLY 
LOCALIZE THE PART. 



THE STRUCTURAL MODEL COMPUTES 

THE PROBABILITY THAT THE PART 
IS PRESENT AND THE BEST LOCATION 
<x2,y2> AND ANGLE THETA3 








STOP AND DECLARE 
PART ABSENT 



STOP AND DECLARE 
PART PRESENT AT 
<x3,y3> AND THETA4 
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BEGIN 




SELECT A MODEL FOR A PART TYPE 



APPLY MODEL TO ALL PLACED 
IMAGES OF THE SAME PART TYPE 



COMPUTE SCORE BETWEEN MODEL|_ 234 
AND EACH ROI PLACED IMAGE 



APPLY MODEL TO ALL PASTE IMAGES 
OF THE SAME PART TYPE 



COMPUTE SCORE BETWEEN MODEL AND 
EACH ROI PASTE IMAGE 



SAVE SCORES 



CHECK FOR OUTLIERS 
(SEE FIG. 14B) 



FROM 253 



TO 244 
F I G. -HB - 



Fter-h4A- 
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